A Decoding-Complexity and Rate-Controlled Video-Coding Algorithm for HEVC

Video playback on mobile consumer electronic (CE) devices is plagued by fluctuations in the network bandwidth and by limitations in processing and energy availability at the individual devices. Seen as a potential solution, the state-of-the-art adaptive streaming mechanisms address the first aspect, yet the efficient control of the decoding-complexity and the energy use when decoding the video remain unaddressed. The quality of experience (QoE) of the end-users’ experiences, however, depends on the capability to adapt the bit streams to both these constraints (i.e., network bandwidth and device’s energy availability). As a solution, this paper proposes an encoding framework that is capable of generating video bit streams with arbitrary bit rates and decoding-complexity levels using a decoding-complexity–rate–distortion model. The proposed algorithm allocates rate and decoding-complexity levels across frames and coding tree units (CTUs) and adaptively derives the CTU-level coding parameters to achieve their imposed targets with minimal distortion. The experimental results reveal that the proposed algorithm can achieve the target bit rate and the decoding-complexity with 0.4% and 1.78% average errors, respectively, for multiple bit rate and decoding-complexity levels. The proposed algorithm also demonstrates a stable frame-wise rate and decoding-complexity control capability when achieving a decoding-complexity reduction of 10.11 (%/dB). The resultant decoding-complexity reduction translates into an overall energy-consumption reduction of up to 10.52 (%/dB) for a 1 dB peak signal-to-noise ratio (PSNR) quality loss compared to the HM 16.0 encoded bit streams

Seamless video access for mobile devices by content-aware utility-based adaptation

Today's Internet multimedia services are characterized by heterogeneous networks, a wide range of terminals, diverse user preferences, and varying natural environment conditions. Heterogeneity of terminals, networks, and user preferences impose nontrivial challenges to the Internet multimedia services for providing seamless multimedia access particularly for mobile devices (e.g., laptops, tablet PCs, PDAs, mobile phones, etc.). Thus, it is essential that advanced multimedia technologies are developed to deal with these challenges. One of these technologies is video adaptation, which has gained significant importance with its main objective of enabling seamless access to video contents available over the Internet. Adaptation decision taking, which can be considered as the "brain" of video adaptation, assists video adaptation to achieve this objective. Scalable Video Coding (SVC) offers flexibility for video adaptation through providing a comprehensive set of scalability parameters (i.e., temporal, spatial, and quality) for producing scalable video streams. Deciding the best combination of scalability parameters to adapt a scalable video stream while satisfying a set of constraints (e.g., device specifics, network bandwidth, etc.) poses challenges for the existing adaptation services to enable seamless video access. To ease such challenges, an adaptation decision taking technique employing a utility-based approach to decide on the most adequate scalability parameters for adaptation operations is developed. A Utility Function (UF), which models the relationships among the scalability parameters and weights specifying the relative importance of these parameters considering video content characteristics (i.e., motion activity and structural feature), is proposed to assist the developed technique. In order to perform the developed adaptation decision taking technique, a video adaptation framework is also proposed in this paper. The adaptation experiments performed using the proposed framework prove the effectiveness of the framework to provide an important step towards enabling seamless video access for mobile devices to enhance viewing experience of users. © 2012 Springer Science+Business Media, LLC

Modeling user perception of 3D video based on ambient illumination context for enhanced user centric media access and consumption

For enjoying 3D video to its full extent, it is imperative that access and consumption of it is user centric, which in turn ensures improved 3D video perception. Several important factors including video characteristics, users’ preferences, contexts prevailing in various usage environments, etc have influences on 3D video perception. Thus, to assist efficient provision of user centric media, user perception of 3D video should be modeled considering the factors affecting perception. Considering ambient illumination context to model 3D video perception is an interesting research topic, which has not been particularly investigated in literature. This context is taken into account while modeling video quality and depth perception of 3D video in this paper. For the video quality perception model: motion and structural feature characteristics of color texture sequences; and for the depth perception model: luminance contrast of color texture and depth intensity of depth map sequences of 3D video are used as primary content related factors in the paper. Results derived using the video quality and depth perception models demonstrate that these models can efficiently predict user perception of 3D video considering the ambient illumination context in user centric media access and consumption environments

3D video bit rate adaptation decision taking using ambient illumination context

3-Dimensional (3D) video adaptation decision taking is an open field in which not many researchers have carried out investigations yet compared to 3D video display, coding, etc. Moreover, utilizing ambient illumination as an environmental context for 3D video adaptation decision taking has particularly not been studied in literature to date. In this paper, a user perception model, which is based on determining perception characteristics of a user for a 3D video content viewed under a particular ambient illumination condition, is proposed. Using the proposed model, a 3D video bit rate adaptation decision taking technique is developed to determine the adapted bit rate for the 3D video content to maintain 3D video quality perception by considering the ambient illumination condition changes. Experimental results demonstrate that the proposed technique is capable of exploiting the changes in ambient illumination level to use network resources more efficiently without sacrificing the 3D video quality perception

3D video bit rate adaptation decision taking using ambient illumination context

3-Dimensional (3D) video adaptation decision taking is an open field in which not many researchers have carried out investigations yet compared to 3D video display, coding, etc. Moreover, utilizing ambient illumination as an environmental context for 3D video adaptation decision taking has particularly not been studied in literature to date. In this paper, a user perception model, which is based on determining perception characteristics of a user for a 3D video content viewed under a particular ambient illumination condition, is proposed. Using the proposed model, a 3D video bit rate adaptation decision taking technique is developed to determine the adapted bit rate for the 3D video content to maintain 3D video quality perception by considering the ambient illumination condition changes. Experimental results demonstrate that the proposed technique is capable of exploiting the changes in ambient illumination level to use network resources more efficiently without sacrificing the 3D video quality perception

A disocclusion replacement approach to subjective assessment for depth map quality evaluation

An inherent problem of Depth Image Based Rendering (DIBR) is the visual presence of disocclusions in the rendered views. This poses a significant challenge when the subjective assessment of these views is utilised for evaluating the quality of the depth maps used in the rendering process. Although various techniques are available to address this challenge, they result in concealing distortions, which are directly caused by the depth map imperfections. For the purposes of depth map quality evaluation, there is a need for an approach that deals with the presence of disocclusions without having further impact on other distortions. The aim of this approach is to enable the subjective assessments of rendered views to provide results, which are more representative of the quality of the depth map used in the rendering process

A disocclusion replacement approach to subjective assessment for depth map quality evaluation

An inherent problem of Depth Image Based Rendering (DIBR) is the visual presence of disocclusions in the rendered views. This poses a significant challenge when the subjective assessment of these views is utilised for evaluating the quality of the depth maps used in the rendering process. Although various techniques are available to address this challenge, they result in concealing distortions, which are directly caused by the depth map imperfections. For the purposes of depth map quality evaluation, there is a need for an approach that deals with the presence of disocclusions without having further impact on other distortions. The aim of this approach is to enable the subjective assessments of rendered views to provide results, which are more representative of the quality of the depth map used in the rendering process